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1. INTRODUCTION 

Multiple rectilinear regressions (MLR) [1] are the foremost common kind of linear regression 
analysis. As a prognostic analysis, it will not justify the link between one continuous dependent variable and 2 
or a lot of freelance variables. The independent variables will be continuous or categorical (dummy coded as 
appropriate). MLR presented in (1) and Figure 1 can be solved using MATLAB Figure 2 [2]-[4]. 


y = do + 1X1 + Q2X3 + + aAnXn (1) 


Where: 
y = dependent variable 
x = explanatory variables 
a = constants 
The coefficients will be as follows: a={5.0, 2.0, 3.5}, So MLR problem can be represented using (2). 


y = 5 + 2x + 3.5x3 (2) 
Artificial neural network (ANN) [5]-[7] is a computational mathematical model, which contains a 
set of fully connected neurons, which are arranged in two or more layers. Each neuron as shown in Figure 3 


implements two operations [8]-[10]: i) finding the sum of products of the inputs and the associated weights 
[11], [12], ii) according to the selected activation function calculate the neuron output. 
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X1 X2 Target output 

2.0000 -3.0000 -1.5000 
3.0000 -4.5000 -4.7500 
-2.0000 -25.0000 -86.5000 
4.0000 0 13.0000 
-5.0000 9.5000 28.2500 
0 10.0000 40.0000 
9.0000 11.0000 61.5000 
10.0000 2.0000 32.0000 
15.0000 3.0000 45.5000 
-11.0000 12.0000 25.0000 
12.5000 -8.0000 2.0000 
18.0000 -2.0000 34.0000 
2.3000 0 49.6000 
4.0000 14.0000 62.0000 
0 5.0000 22.5000 
-6.8000 11.0000 29.9000 


Figure 1. MLR example 


>> xl=[2 3 -2 4 -5 0 9 10 15 -11 12.5 18 22.3 4 0 -6.8]; 
x2=[-3 -4.5 -25 0 9.5 10 11 2 3 12 -8 -2 0 14 5 11] 

yy= (5+2*x14+3 .5*x2) ; 

X = [ones (size(xl"}} xl" x2"]; 

a = X\yy' 


Figure 2. MATLAB code to solve MLR 
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Figure 3. Neuron operations [13], [14] 


Different types of ANN can be used to solve MLR problem, the first one is feed forward ANN with 
back propagation (FFANN), in this ANN as shown in Figure 4 each neuron in any layer are fully connected 
to the previous inputs [15]-[17]. The second type is cascade feed forward ANN (CFANN) [18], [19], in this 
type of ANN the inputs are used to feed the hidden layer as shown in Figure 5. The last type which will be 
investigated in this paper is Elman ANN (EANN), in this type as shown in Figure 6 a context inputs formed 
from the hidden layer are used as inputs [20], [21]. 

To use any type of ANN to solve MLR problem we have to follow the steps [22]-[24]: i) select the 
input data set, which is the values of explanatory variables; ii) select the target, which is the values of 
dependent variable; iii) create ANN, here we define the type of ANN, ANN architecture: How many layers 
are to be used, and how many neurons in each layer. In this step we have to define activation function for 
each layer, logsig or tansig for the input layer and the hidden layers, linear activation function for the output 
layer [25]; iv) initialize the weights; v) set the error parameter to zero; vi) select the number of training 
cycles; vii) train ANN; viii) save ANN, ix) run ANN to solve MLR problem. 
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Figure 6. EANN 


2. RESEARCH METHOD 

Different types of ANN [11]-[13] with different architectures were used to solve MLR problem, 
some experiments were tested, and below we will investigate the results of these experiments: 
Experiment 1: 

Here we select a two layers ANN, input layer with two neurons and output layer with one neuron, 
Table 1 shows the obtained results of using various types of ANN: maximum error was calculated using (3). 


MaxErr = max(abs(y, — y)) (3) 
Figure 7 shows the obtained outputs for various ANN. Figures 8, 9, and 10 shows the error between 


the target and the calculated output of each ANN. From the obtained results here in this experiment we can 
see that CFANN has the best performance and we can recommend it to solve MLR problem. 


Table 1. Experiment 1 results 


ANN type Number of layers Neurons Activation function Max. Error Training cycles 
FFANN 2 2,1 Logsig, linear 0.0079 10000 
CFANN 2 2,1 Logsig, linear 0 9 

EANN 2 2,1 Logsig, linear 31.1152 10000 
FFANN 2 2,1 Tansig, linear 0.0001059 10000 
CFANN 2 2,1 Tansig, linear 0 10 

EANN 2 2, 1 Tansig, linear 24.7341 10000 
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Target CF output FF output 
-1.5000 -1.5000 -1.5000 
-4.7500 -4.7500 -4.7500 

-86.5000 -86.5000 -86.5000 
13.0000 13.0000 13.0000 
28.2500 28.2500 28.2500 
40.0000 40.0000 40.0000 
61.5000 61.5000 61.5000 
32.0000 32.0000 32.0000 
45.5000 45.5000 45.5000 
25.0000 25.0000 25.0000 

2.0000 2.0000 2.0000 
34.0000 34.0000 34.0000 
49.6000 49.6000 49.6000 
62.0000 62.0000 62.0000 
22.5000 22.5000 22.5000 
29.9000 29.9000 29.9000 


ELM output 
-2.1458 
-5.2117 
-61.0822 
14.9681 
42.5609 
47.1758 
50.1927 
37.1704 
45.3666 
-5.3184 
-4.7442 
33.4190 
45.0598 
50.4441 
33.4453 
29.3088 
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Figure 7. Obtained outputs for each ANN 
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Figure 9. Calculated error for FFANN 


Experiment 2: 
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Figure 10. Calculated error for EANN 


The architectures of ANNs here were changed to see how changing ANN architecture will affect the 
performance, Table 2 shows the experimental results. From Table 2 we can see that changing ANN 
architecture does not positively affect the performance; hence we can conclude that using CFANN with two 
layers is best solution of MLR problem, here the number of neurons in the input layer must equal to the 
number of explanatory variables. Figure 11 shows the calculated outputs of using CFANN to solve MLR 
problem with 4 explanatory variables (here the number of neurons in the input layer 4), while Figure 12 
shows the error between the target and the calculated output. 


Table 2. Experiment 2 results 


ANN type Number of layers Neurons Activation function Max. Error Training cycles 
FFANN 3 2,4,1 Tansig, tansig, linear 0.00014588 10000 
CFANN 3 2,4,1 Tansig, tansig, linear 0 796 

EANN 3 2,4,1 Tansig, tansig, linear 55.5833 10000 
FFANN 3 2,4, 1 logsig, tansig, linear 0.00013311 10000 
CFANN 3 2,4,1 logsig, tansig, linear 0 1171 

EANN 3 2,4, 1 logsig, tansig, linear 25.0369 10000 
FFANN 3 2,4, 1 logsig, tansig, linear 0.0048 10000 
CFANN 3 2,4,1 logsig, tansig, linear 0 114 

EANN 3 2,4, 1 logsig, tansig, linear 58.0207 10000 
FFANN 3 2,4, 1 Tansig, logsig, linear 0.0034 10000 
CFANN 3 2,4,1 Tansig, logsig, linear 0 317 

EANN 3 2,4, 1 Tansig, logsig, linear 29.515 10000 
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Figure 11. Results of MLR with 4 variables 
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Figure 12. Calculated error for CFANN 


3. CONCLUSION 

Different types of ANNs with different architectures were used to solve MLR problem. Each ANN 
was treated with various input data set varying the activation function. It was seen that CFANN has the best 
performance, and here we can highly recommend CFANN to solve MLR problem with any number of 
explanatory variables by selecting two layers and adjusting the number of neurons in the input layer to match 
the number of explanatory variables in MLR problem. Provide a statement that what is expected, as stated in 
the "Introduction" chapter can ultimately result in "Results and Discussion" chapter, so there is compatibility. 
Moreover, it can also be added the prospect of the development of research results and application prospects 
of further studies into the next (based on result and discussion). 
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